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ABSTRACT 

This study investigated the attitudes of primary school students towards computer 
based testing and assessment in terms of different variables. The sample for this 
research is primary school students attending a computer based testing and 
assessment application via CITO-OiS. The "Scale on Attitudes towards Computer 
Based Testing and Assessment" to collect data and the results obtained were 
compared in terms of school type, gender, and grade level. 

The results of this study revealed that significant differences exist between 
attitudes of students from different schools. Such a difference does not exist 
between attitudes of students when their genders, grade levels, and participation 
periods to computer based assessment are taken into account. 
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INTRODUCTION 

Recent developments in computer technologies have influenced many areas 
including educational testing and assessment. It is possible to group those effects 
under two titles. First, improvements in computer technologies have enabled more 
complicated statistical applications for test and scale development, which in turn 
have enabled discussion and development of more advanced statistical analyses. 

The item response theory, which was derived against the true score theory and the 
confirmatory factor analysis, which is used in addition to exploratory factor analysis 
to test the construct validity during scale development could be set as examples. 
Second, those advancements in computer technologies have led to the development 
of web-based, computer-based, and/or computerized adaptive test applications, 
besides those traditional paper-pencil tests. 

As now computer-based tests are started to be used especially at primary education 
level, for such reasons like their test results are scored instantly and reported in 
detail, their applications are flexible, effective and reliable, they minimize costs for 
long-term applications, they enable usage of different item types compared to 
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traditional paper-pencil tests, along with, they make the integration of audio-visual 
materials possible and enable the rater to easily adjust the student response time 
(Wang, Jiao, Young, Brooks & Olson, 2008, Bugbee, 1996; Drasgow & Olsen- 
Buchanan, 1999; Gvozdenko & Chambers, 2007; McKee, & Levinson, (1990); Mead 
& Drasgow, 1993; Parshall, Spray, Kalohn, & Davey, 2002; Smith 8i Caputi, 2005; 
Thelwall, 2000; Tseng, Macleod, & Wright, 1997). On the other hand in many 
countries, driving tests, military training tests, personnel staffing exams in private 
sectors, and certificate tests applied by vocational groups can be listed as 
computer-based test examples (Russo, 2002; Trotter, 2001). Similarly, though not 
common, such computer based testing and assessment (CBA) applications have 
started to be utilized in Turkey. It can be shown that such as TTnet-Vitamin, Pisa 
and Cito-Ois (student monitoring system) applications are examples. 

CBA applications have brought up some questions. There are many studies abroad 
on this issue (Choi & Tinkler, 2002; Kingston, 2009; Kim, 1999; McKee, L. M., & 
Levinson, E. M., 1990; Mead and Drasgow, 1993; Neuman, G., & Baydoun, R., 1998; 
Pomplun, M., & Custer, M., 2005). Those studies mainly focus on comparisons 
between paper-pencil tests (PPTs) and computer-based tests (CBTs). In their meta¬ 
analysis Mazzeo and Harvey (1988), investigated the research that focused on 
paper-pencil and computer-based intelligence, aptitude, personality and 
achievement tests. 

The analyses revealed varying results, but showed that computer-based 
applications increased the response time compared to paper-pencil ones. 
Additionally, the studies (Choi & Tinkler, 2002; Kim, 1999; Kingston, 2009; Mead & 
Drasgow, 1993; Peak, 2005) done subsequently revealed no significant differences 
in achievement considering computer-based and paper-pencil exams; however 
taking the academic content and grade levels into consideration, it was observed 
that students had hard times with the CBTs. Leeson (2006), identifies the factors 
lead to difficulties in CBA applications under two titles, as factors originating from 
"users" and "technology used". He states that the user's gender, his/her ability to 
process information, ability to use a computer, and his/her level of anxiety could 
have an influence on an application, whereas he gives the size and resolution of 
monitors, writing character and its length, the way the problem is presented, and 
having the option of review or not as technology originated factors. Many 
researchers have already done studies investigating the relationship between 
computer usage ability and achievement. Some of them (Goldenburg & Pedulla, 
2002; Pomplun & Custer, 2005, Pomplun, Ritchie & Custer, 2006, Bennett, Braswell, 
Oranje, Sandene, Kaplan, & Yan, 2008) have stressed that computer usage ability is 
an important predictor of respondent achievement, therefore those students poor 
at computers may show low achievement in CBAs; however they add that with the 
increase in computer technologies and access opportunities, such problems may 
decrease. 

In some of the studies investigating the difference between methods in terms of 
gender, race and age (Bennett et al., 2008; Clariana & Wallace, 2002) no significant 
difference was found in achievements, whereas in some other studies (Gallagher, 
Bridgeman, & Cahalan, 2000) little significant difference was observed. In their 
recent study Terzis and Economides (2011) describe the trends of male and female 
students towards CBAs. 
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As a result, they highlight that both genders have positive views on CBAs if they are 
based on games, are open and course-related. Additionally they state that the 
attitudes of boys and girls towards CBAs are under the influence of social 
environment; the boys focus on the usefulness while the girls stress the ease of 
use. 

Some other studies investigating the "technology used", focused on the influence of 
the size and resolution of the monitor and characters on applications. McKee and 
Levinson (1990) state that such traits might vary according to the nature of the 
area being assessed. Additionally, Lunz (1995) and Vispoel (2000) highlight the 
present flexible applications inherent in PBTs (like reviewing the questions, starting 
from any question you'd like or skipping a question, etc.) are not observed in CBAs, 
therefore this can have negative effects on achievement (Wang at all. 2007). 

The fact that various researches have different results can be due to 
methodological problems in those studies and if effects of PPTs and CBAs are to be 
compared, equivalent tests should be used in each application (Wang at all. 2007). 
The analysis of some recent studies (Cheung, Lee, & Chen, 2002; Joosten-ten 
Brinkeet all., 2007; Kaklauskas et all., 2010; Kesici, Sahin, & Akturk, 2009; Smith & 
Caputi, 2007; Terzis and Economides, 2011; Wang et all., 2008; Yuen & Ma, 2002) 
show that they focus on gender differences, e-learning acceptance, social influence, 
computer self-efficacy, computer anxiety and equivalency of computer-generated 
and paper-pencil scores. 

One potential source of validity error that needs to be considered in computerized 
testing is computer anxiety. A number of researchers have discussed the 
importance of behavioral and cognitive aspects of computer anxiety as a potential 
source of interference in CBA administration (Deane et al., 1995b; Glass & Knight, 
1988; Schulenberg & Yutrzenka, 1999; Shermis & Lombard, 1998; Tseng et al., 
1998). On the other hand, according to Terzis and Economides (2011), researchers 
have realized that there are differences between men and women regarding their 
perceptions and effects on the relationships among the constructs that affect the 
behavioral intention to use computers and e-learning. However, the literature did 
not provide conclusive results. Concerning perceptions and attitudes, earlier studies 
showed that male students were more positive towards computer use in a learning 
context. In addition, their research have aimed to explore gender differences in 
perceptions and relationships among dominants affecting computer based 
assessment acceptance, as well as how gender influences learners' attitudes 
towards CBAs, which factors are affecting the use of them, and why each gender 
would use it. The results of study have shown that the male students have been 
tempted to use CBAs for their playfulness, usefulness, content, and social influence. 
On the other hand, female students have been mainly tempted to use CBAs for their 
playfulness, ease of use, content, and goal expectancy. Thus, playfulness and 
content are also important for females as well, but not to the same degree for 
males. However, female students' behavioral intention is defined also by ease of 
use and goal expectancy, not by usefulness and social influence as in the case of 
males. 

It is seen that the related studies found in literature on CBAs have various results. 
However, there is a common perception that students approach such applications 
positively. 

179 



At the same time, the same topic is associated with computer skills and computer 
anxiety. Besides, apart from a few studies (TTNet-Vitamin, Pisa) in Turkey, one of 
the new applications is CITO (cache in Trash Out) OiS (Student Monitoring 
System). This application aims to provide feedback to students, teachers, schools, 
parents, and the Ministry of National Education (MONE) on students' difficulties in 
basic and sub-areas and their skills to be improved. The application provides 
teachers this information both on student and class level. By monitoring the needs 
of students for several attainments in various areas, teachers could prepare extra 
activities and tasks. Schools receive feedback on assessments of every students' 
attainments, the questioning of school performance in terms of the norms in 
Turkey, and taking measures on raising the quality standards; whereas parents 
have the opportunity to learn about the levels and developments of the kids 
(Berberoglu, 2011). 

Since CITO-OiS is an application of computer based testing and assessment, the 
views and attitudes of students towards it is an issue of interest. The present study 
aims to identify the attitudes of those primary education level students (3rd, 4th, 
and 5th grades) participating CITO-OiS towards CBA and analyze their perceptions 
in terms of grade, gender, and school type. 

METHOD AND PARTICIPANTS 

This descriptive research aims to explore and compare the attitudes of students 
towards CBA in terms of various variables. The population is all the schools and 
students participating in the CITO-OiS applications. There are 48 school in Turkey, 
35 are private schools and 13 are state schools participating in this applications. For 
research purposes, layered group sampling was administered. Layers are types of 
schools (private or state school), groups are schools. Six schools (3 private and 3 
state) were chosen as samples. 784 students (390 female, 394 male) from those 
sampled schools participated. 631 of those students enroll at private schools, 153 
of them are from state schools. 268 of them are primary education 3rd graders, 224 
are 4th graders, and 292 are 5th graders. 

Instrumentation 

The data collection instrument is a 35-item scale developed by the authors 
themselves and entitled as "Scale of Attitudes towards Computer Based Testing and 
Assessment". The following stages were followed during the scale development 
process: First, an item pool comprising of 71 statements related to computer based 
testing and assessment was formed, which was later presented to the views of 
experts from various areas like Computer and Instructional Technologies, 
Curriculum and Instruction, and Educational Measurement and Evaluation, 
eventually ended up with 54 items ready for pre-trial. The pre-trial from was given 
to 440 students and statistical reliability and validity analyses were done on those 
results. By implementing exploratory factor analysis, the construct of the scale was 
tested, items with high cross-loadings and items that did not load strongly (below 
.40) on any factor were deleted (19 items), and eventually those items loading one 
factor were included in the final form of the scale (35 items). 

The factor has an Eigen value of 17.08, explains 50.24% of the variance, and item 
factor loadings range from .49 to .83. The reliability coefficient calculated by alpha 
internal consistency method (Yurdabakan, 2008) is .97. 
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FINDINGS 


The Distribution of Students' Opinions towards CBA 

In this section, the mean scores and standard deviations of the items from the 
whole scale were calculated. The results (the highest and the lowest 10 items) are 
presented in table 1. Later, students' attitudes were analyzed in terms of gender, 
school type (private and state), and grade levels (3rd, 4th, and 5th). 

Table: 1 

Distribution of Scale Item Mean Scores and Standard Deviations 


Items 

X 

Sx 

11. It works our brains. 

3.82 

1.37 

12. It improves me. 

3.81 

1.38 

24. The report of this assessment shows me what I know 

3.79 

1.49 

or don't. 

7.1 think it is a useful application. 

3.78 

1.39 

8. I think it measures my knowledge level. 

3.76 

1.37 

32. People would love it if it was made more entertaining. 

3.75 

1.43 

26.1 learn my weaknesses. 

3.73 

1.45 

18. My teachers and parents learn about my weaknesses 
together with me. 

3.69 

1.41 

2. Results show my learning level. 

3.66 

1.40 

16. It is more fun for me to do it with earphones. 

3.65 

1.49 

30.1 don't want this application. 

2.43 

1.54 

31.1 think it is a bit silly and boring. 

2.37 

1.48 

10.1 don't think it has any use for me. 

2.36 

1.55 

4. Doing the items on computer distracts me. 

2.25 

1.46 

1. Questions on computer seem silly to me. 

2.21 

1.40 

28.1 find it useless. 

2.21 

1.45 

21. It is too boring for me. 

2.20 

1.44 

6. It is too difficult for me. 

2.11 

1.39 

19.1 don't take it seriously because it seems like a game. 

2.01 

1.42 

14. It weans me off from the courses. 

2.00 

1.35 


The analysis of Table 1 reveals that the means and Standard deviations of item 
responses range between 3.82±1.37; 2.00±1.35. Students agreed mostly with 
positive items reflecting CBA, such as "It works our brains", "It improves me", "The 
report of this assessment shows me what I know or don't", "I think it is a useful 
application", and "I learn my weaknesses". On the other hand, they agreed less on 
negative items like "It weans me off from the courses", "I don't take it seriously 
because it seems like a game", "It is too difficult for me", "It is too boring for me", 
"I find it useless", "Questions on computer seem silly to me", "Doing the items on 
computer distracts me", and "I don't think it has any use for me". 

The distribution of means and Standard deviations show that students agreed with 
positive items, whereas they showed little agreement with the negative ones. 
Depending on these results, it can be said that students generally have positive 
views on CBA applications. . „ . 






The Comparison of Attitudes in Terms of Gender, School Type, Grade Level and 
Participation Period to CBA 

To analyze students' attitudes towards CBA in terms of gender and school type, the 
mean scores were compared with unrelated/uncorrelated t-test statistics and 
results are presented in Table: 2. 


Table: 2 

The Comparison of Student Attitudes According to Genders and School Type 



Group 

N 

X 

S x 

df 

t 

P 

Gender 

Female 

Male 

390 

394 

3.06 

3.12 

.39 

.42 

782 

-1.88 

.06 

School 

Private 

State 

631 

153 

3.07 

3.18 

.41 

.39 

782 

-2.70 

.007** 


**P<.01 


The analysis of results in Table: 2 reveal that, there are no significant differences 
between student attitudes towards CBA in terms of gender, whereas there is a 
significant difference between the attitudes of private school students and state 
school students, where the difference is in favor of students enrolling at state 
schools. These results show that in terms of gender there are no differences 
between the attitudes of males and females; however differences exist between the 
attitudes of students enrolling at private and state schools. Another comparison 
was made to analyze the effects of grade levels and periods of participation 
(experience) on attitudes. First, the attitudes of students enrolling at different 
grade levels were compared using variance analysis. 

Results are presented in Table: 3. 

Table: 3 

The Comparison of Student Attitudes According to Grade Levels 


Grade 

n 

X 

Source 

KT 

DF 

KO 

F P 

3 

268 

3.08 

Within 

Groups 

.484 

2 

.242 

1.454 .23 

4 

224 

3.07 

Between 

Groups 

129.89 

781 

.166 





Total 

130.38 

783 



5 

292 

3.12 






Total 

784 

3.09 







The results in Table: 3 show that, the means of 3rd and 4th graders are close, 
whereas the means of 5th graders are a bit higher than the others. But the variance 
analysis employed exhibits no significant difference among groups. A similar 
comparison was made considering the period of participation to the application, in 
other words, students' experience with the application. 
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For research purposes, students were divided into three groups and their mean 
scores were tested using variance analysis. The results are presented in Table 4. 


Table: 4 

The Comparison of Student Attitudes According to Period of 
Participation to Application 


Period 

n 

X 

S x 

Source 

KT 

DF 

KO 

F 

P 

New 

161 

3.14 

.39 

Within 

Groups 

.413 

2 

.20 

6 

1.24 

.29 

1 year 

154 

3.07 

.35 

Between 

Groups 

129.97 

781 

.16 

6 



1 year> 

469 

3.08 

.43 

Total 

130.38 

783 




Total 

784 

3.09 

.41 



The variance analysis results show that there is no significant difference among 
scores, even though the new beginners have slightly higher scores. In other words, 
the periods of participation to the CITO-OiS application have no meaningful effects 
on attitudes. 

RESULTS AND DISCUSSION 

The advancements in computer technologies have led to several innovations in 
many areas, including educational testing and assessment. These innovations 
increased opportunities both in test and scale development and application. Among 
those newly developed computerized educational testing and assessment practices 
are computerized tests, adaptive tests, computer based assessments and web- 
based assessments. Although this issue has gained importance in the last 30 years, 
it is still new in Turkey. One of the most comprehensive web-based or computer 
based assessment applications in Turkey is CITO-OIS. The present study aims to 
identify the perceptions of Turkish students towards such applications. For research 
purposes, the attitudes of those students participating in the CITO-OIS applications 
towards CBA were identified and investigated thoroughly in terms of several 
variables. 

According to the results of descriptions and comparisons, it can be said that 
students generally present positive attitudes (X=3.09±.41) towards CBAs. This 
result parallels the results in literature (Bernard, 1997; Terzis and Economides, 
2011). On the other hand, considering students gender and schools they enroll at, 
no significant difference was found between the attitudes of boys (X=3.12±.42) 
and girls (X=3.06±.39). It is possible to state that male and female students 
evaluate CBAs at similar levels. Although the studies in literature mostly focus on 
comparing the traditional paper-pencil tests and CBA applications, as authors like 
Bernard (1997) and Terzis and Economides (2011) put forward, such a finding is 
parallel to studies in which perceptions towards CBAs are compared. Another result 
of the present study is the difference found between the attitudes of students in 
state and private schools, where the difference was in favor of state schools. 
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Though no other study on this issue is present in literature, it is possible to explain 
this topic in terms of computer accession opportunities. 

Even though computer accession opportunities increase students' computer 
competencies and CBA achievements (Goldenburg & Pedulla, 2002; Pomplun & 
Custer, 2005, Pomplun, Ritchie & Custer, 2006, Bennett, at all. 2008), it is possible 
to evaluate that such approaches could be the reason of students' limited accession 
opportunities. Similarly Terzis and Economides (2011) stress that, besides many 
other factors, attitudes towards CBA applications are under influence of social 
environment. On the other hand, the variance analyses done to compare the 
attitudes towards CBAs in terms of grade levels and periods of participation yielded 
no significant differences. However, 5th graders and those new starters had slightly 
higher mean scores. The case with the upper grades could be attributed to 
experience (Goldenburg & Pedulla, 2002; Pomplun & Custer, 2005, Pomplun, 
Ritchie & Custer, 2006, Bennett, at all. 2008), whereas the case with the new 
starters could be attributed to the excitement of doing something different from 
ordinary applications. Still, it would be useful to have further thorough studies on 
this issue. 

As a result, this research analyzed the attitudes of primary education students 
towards computer or web-based testing and assessment applications in Turkey in 
terms of several variables. This topic is not limited to analysis of attitudes. On the 
contrary, especially in subsequent research it can be useful to study the influence of 
such practices on student achievements, customs in learning and assessment, and 
more importantly, on test development and application opportunities. 

Notes from Author's: This study was adapted from the unpublished Master's Thesis 
Report (Uzunkavak, C.), Dokuz Eylul University, Faculty of 
Education, 2012, Turkey. 
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APPENDIX A 


1. Could you tell me a little about yourself? 

> What grade are you in? 

> How old are you? 

> Where do you live? 

2. Could you tell me a little about your school? 

> What part of the province are you in? 

> What kind of school is it (i.e., what grades does it include)? 

> Roughly how many students are in your school? 

> Roughly how many communities does your school in take from? 

3. C 
ould you tell me about the web-based courses have you taken? 

> How many? 

> Which ones? When? 

4. T 
hink about a typical day when you've had both online and offline classes. 
Describe for me what would look like for you. 

5. What do you like about your web-based course(s)? What do you dislike? Why? 

6. If you could change something(s) about your web-based course, what would it 
(they) be? Why? 

7. W 
hat do you miss about face-to-face classes when you are in a web-based 
course? What do you miss about web-based courses when you are in a face-to- 
face class? 

> What is the difference in work load outside of class for you? 

> Which is more? Why do you think that is? 

8. O 
ne of the issues related to your web-based classes raised in the surveys was 
that students felt that lack of time was one of the main problems. What has 
your experience been with the amount of time it takes in your web-based 
classes? 

9. W 
hen this survey was conducted using university students, one of the issues they 
raised related to the notion of community or feeling like you have connections 
with others when you are in class together. First, do you think that this is an 
important aspect in a school experience? Can you talk to us about your sense of 
connections with others in a face-to-face classroom versus a web-based course? 

10. W 
hat suggestions would you give to students who are taking web-based courses 
for the first time? What suggestions would you give to the designers of your 
web-based courses? What suggestions would you give to the teachers of your 
web-based courses? 
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